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SYSTEM AND METHOD FOR EXTRACTING DEMOGRAPHIC INFORMATION 



2 

3 CROSS-REFERENCE TO RELATED OR CO-PENDING APPLICATIONS 

4 This application relates to co-pending U.S. Patent Application PDNo. 

5 200300184 Serial No. 10/645210, entitled "Selective Sampling For Sound Signal 

6 Classification," filed on August 21, 2003, by Xiaofan Lin; U.S. Patent Application 

7 PDNo. 200309365 Serial No. 10/699264, entitled " System And Method For Call 

8 Center Dialog Management," filed on October 30, 2003, by Sherif Yacoub; and U.S. 

9 Patent Application PDNo. 200309899 entitled " System And Method For Language 

10 Variation Guided Operator Selection," filed on January 30 2004, by Lin et al. These 

11 related applications are commonly assigned to Hewlett-Packard of Palo Alto, CA. 
12 

13 BACKGROUND OF THE INVENTION 

14 1. Field of the Invention 

15 The present invention relates generally to systems and methods for call 

16 handling, and more particularly for extracting demographic information. 

17 2. Discussion of Background Art 

18 Automated call handling systems, such as Interactive Voice Response (IVR) 

19 systems, using Automatic Speech Recognition (ASR) and Text-to-speech (TTS) 

20 software are increasingly important tools for providing information and services to 

21 contacts in a more cost efficient manner. IVR systems are typically hosted by a server 

22 that includes an array of Digital Signal Processors (DSPs) and enable contacts to 

23 interact with corporate databases and services over a telephone using a combination of 

24 voice utterances and telephone button presses. IVR systems are particularly cost 

25 effective when a large number of contacts require data or services that are very similar 

26 in nature, such as banking account checking, ticket reservations, etc., and thus can be 
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1 handled in an automated manner often providing a substantial cost savings due to a 

2 need for fewer human operators. 

3 Knowledge of a contact's demographic characteristics within a call center 

4 however would be very valuable. Such demographic information enables IVR 

5 systems to make smarter decisions when providing information to contacts. For 

6 instance, advertisements are preferably targeted to a demographically well-defined 

7 group of people (e.g. young adults, woman under 50, or retired people). However, 

8 directly prompting contacts for such information is typically not desirable and so 

9 currently advertisements are not very demographically specialized. A contact's 

10 demographic information is also useful for tailoring the IVR system's responses to the 

11 contact's characteristics, such as avoiding fancy prompts with a tense contact or 

12 selecting the contact's gender as the synthetic voice generated by the IVR system. 

13 Similarly, the vocabulary and jargon used by the IVR system could also be adapted to 

14 the contact, and stressed callers (i.e. "contacts") could be considered when selecting 

15 an operator to handle the contact's call. 

16 Accurately identifying a contact's demographic characteristic, however, is 

17 actually a very difficult problem. Many current systems for demographic 

18 classification use acoustic classifiers. Acoustic classifiers extract voice features from 

19 a contact's speech signal in an attempt to distinguish one or more of a contact's 

20 demographic characteristics, such as gender, age, accent, emotional state, etc. 

21 However, acoustic classifiers often have such a high error rate that many IVR systems 

22 will not deploy them. For instance, a company that repeatedly presents a 

23 demographically incorrect type of information to a contact, such as playing male- 

24 targeted ads for females, will reflect poorly on a contact's perception of that company. 
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1 In response to the concerns discussed above, what is needed is a system and 

2 method for extracting demographic information that overcomes the problems of the 

3 prior art. 
4 
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1 SUMMARY OF THE INVENTION 

2 The present invention is a system and method for extracting demographic 

3 information. The method of the present invention includes the elements of: initiating 

4 a dialog between a contact and a call handling system; selecting a set of demographic 

5 characteristics; assigning a set of acoustic confidence scores to the demographic 

6 characteristics; assigning a set of substantive confidence scores to the demographic 

7 characteristics; combining the acoustic and substantive confidence scores for each of 

8 the demographic characteristics; and tailoring information presented to the contact 

9 using the set of combined confidence scores. 

10 The system of the present invention includes: an Interactive Voice Response 

11 (IVR) module for initiating a dialog between a contact and a call handling system, and 

12 selecting a set of demographic characteristics; an acoustic classifier for assigning a set 

13 of acoustic confidence scores to the demographic characteristics; a substantive 

14 classifier for assigning a set of substantive confidence scores to the demographic 

15 characteristics; and a data combiner for combining the acoustic and substantive 

16 confidence scores for each of the demographic characteristics; wherein the Interactive 

17 Voice Response module further tailors information presented to the contact using the 

18 set of combined confidence scores. The system also includes all means and 

19 embodiments for effecting the method. 

20 These and other aspects of the invention will be recognized by those skilled in 

21 the art upon review of the detailed description, drawings, and claims set forth below. 
22 
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1 BRIEF DESCRIPTION OF THE DRAWINGS 

2 Figure 1 is a dataflow diagram of one embodiment of a system for extracting 

3 demographic information; 

4 Figure 2 is a flowchart of one embodiment of a root method for extracting 

5 demographic information; and 

6 Figures 3A and 3B are a flowchart of one expanded embodiment of the root 

7 method for extracting demographic information. 
8 
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1 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

2 The present invention provides multiple method combinations for extracting 

3 demographic information, such as information pertaining to the contact's gender, age, 

4 accent, smoker-nonsmoker status, mental tension or stress, being under the influence 

5 of psychotropic substances, etc. This information is gleaned from a contact's 102 

6 utterances to a call handling system 104, such as a call center. The demographic 

7 information is then used to automatically adjust information presented to the contact 

8 102. The methods for extracting demographic information include not only methods 

9 that operate on a speech signal level, but also methods that engage the contact 102 in a 

10 dialog. Reliance on more than one demographic extraction method provides a greater 

11 demographic accuracy that otherwise could be obtained from only one method by 

12 itself. 
13 

14 Figure 1 is a dataflow diagram of one embodiment of a system 100 for 

15 extracting demographic information. The call handling system 104 of the present 

16 invention preferably provides a voice interactive information management service to a 

17 set of contacts. Anticipated information services include those associated with 

18 customer response centers, enterprise help desks, business generation and marketing 

19 functions, competitive intelligence methods, as well as many others. Contacts may be 

20 customers, employees, or any party in need of the call handling system's services. 

21 To begin, a dialog between the contact 102 and the call handling system 104 is 

22 initiated. The dialog begins once a call manager 106 connects the contact 104 to an 

23 Interactive Voice Response (IVR) module 108. The IVR module 108 provides an 

24 automated interface between the contact's 104 speech signals received over a voice 

25 channel and the system's 100 overall functionality. To support such an interface with 

26 the contact 104, the IVR module 108 may include a Text-To-Speech (TTS) translator, 
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1 Natural Language Processing (NLP ) algorithms, Automated Speech Recognition 

2 (ASR), and various other dialog interpretation (e.g. a Voice-XML interpreter) tools. 

3 The IVR module 108 retrieves and presents predetermined dialog segments 

4 from a dialog database 1 10 to the contact 102. The IVR module 108 receives 

5 responses and information requests back from the contact 102 which are then stored in 

6 a contact database 112. ' 

7 Knowledge of the contact's 102 demographic information in many cases, 

8 however, enables the IVR module 108 to retrieve and present a more tailored set of 

9 dialog segments to the contact 102. Such tailored dialog segments typically enables 

10 the dialog to proceed more smoothly to the benefit of both the contact 102 and the 

11 call-handling system 104. Thus the IVR module 108 activates a set of demographic 

12 classifiers, including an acoustic classifier 114 and a set of substantive classifiers 115. 

13 The substantive classifiers 115 include a probing dialog classifier 1 16, and a multiple 

14 choice classifier 1 18. The substantive classifiers 1 15 are herein defined as a set of 

15 classifiers that use the dialog's words, sentences, context, and so on in order to 

16 identify the contact's 102 demographic characteristics. The substantive classifiers 115 

17 can deduce various of the contact's 102 demographic characteristics from the dialog 

18 using various active methods such as question and answer exchanges with the contact. 

19 The substantive classifiers 115 can alternatively deduce such demographic 

20 characteristics by passively observing the information that the contact 102 requests 

21 and provides in the course of the dialog. In some cases, the results from the 

22 substantive classifiers 1 15 are not independent and so the results from substantive 

23 classifier can be fed into the other to improve classification accuracy. 

24 These classifiers generate confidence scores indicating a probability that the 

25 contact 102 has a particular set of demographic characteristics. These confidence 

26 scores are continuous in nature, ranging from p=0.0 to p=L0, and can be combined 
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1 into an overall confidence score. These classifiers preferably do not output a binary 

2 yes-no output. 
3 

4 Acoustic Classifier 

5 The call manager 106 diverts a copy of the contact's 102 speech utterances to 

6 the acoustic classifier 1 14 without interrupting the dialog between the contact 102 and 

7 the IVR module 108. 

8 The acoustic classifier 1 14 extracts from the contact's 102 speech signal 

9 features relating to one or more demographic characteristics that need to be resolved 

10 with respect to the contact 102. Those skilled in the art recognize various speech 

11 signal features that are of use in identifying various demographic characteristics (i.e. 

12 pitch and cepstral features are useful in gender determination). 

13 The acoustic classifier 114 compares the contact's 102 speech signal features 

14 to a predefined body of speech signal features generated using neural net techniques 

15 and assigns a set of confidence scores to each of the demographic characteristics. The 

16 acoustic classifier 114 stores the confidence scores in a demographic data 120 portion 

17 of the contact database 1 12. 

18 An example of an acoustic classification is now discussed. For the purposes of 

19 this example, the contact's 102 gender is initially inferred just from the confidence 

20 scores assigned by the acoustic classifier 1 14. So if the contact 102 is a woman with a 

21 deep voice, the acoustic classifier's 1 14 confidence scores might indicate a high 

22 probability that the contact 102 is almost equally likely to be a female (p=0.9) or a 

23 male (p=0.8). However, if the contact 102 is a man with a deep voice, the acoustic 

24 classifier's 1 14 confidence scores might indicate a low probability that the contact 102 

25 is a female (p=0. 1), but a very high probability that the contact 102 is a male 

26 (p=0.999). Thus for the male with a deep voice, there is only a 0.1% chance of 
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1 assigning gender incorrectly. However, for the female with a deep voice there is a 

2 10% chance of assigning the gender incorrectly. In this latter case, the IVR module 

3 108 would request additional help from the other classifiers 1 16 and 1 18 to resolve the 

4 contact's 102 gender. 
5 

6 Probing Dialog Classifier 

7 A probing dialog classifier 1 16 presents the contact 102 with one or more 

8 probing dialogs retrieved from the dialog database 110. The probing dialogs ask a set 

9 of questions related to one or more demographic characteristics that need to be 

10 resolved with respect to the contact 102. The probing dialog classifier 116 compares 

11 the contact's 102 responses to a predefined body of responses generated using neural 

12 net techniques, however, other pattern recognition classifiers are also possible. Then 

13 the probing dialog classifier 1 16 assigns a set of confidence scores to each of the 

14 demographic characteristics. The probing dialog classifier 1 16 stores the confidence 

15 scores in the demographic data 120 portion of the contact database 112. 

16 An example of a probing dialog is shown below. In this example, the IVR 

17 module 108 needs to know the contact's 102 gender before responding to the contact's 

18 movie theater inquiry. For the purposes of this example, the contact's 102 

19 demographic characteristics are inferred just from the confidence scores assigned by 

20 the probing dialog classifier 116. 

21 IVR module: Please hold while we retrieve information about movies in the 

22 theater you selected. 

23 ("Probing Dialog" Starts Here) 

24 IVR module: While holding, are you interested in receiving promotions on 

25 "cosmetics"? 

26 Contact: No 
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1 IVR module: Would you be interested in some current promotions on 

2 "workshop tools"? 

3 Contact: Yes 

4 ("Probing Dialog" Ends Here) 

5 (Demographically Tailored Advertisement Starts Here) 

6 IVR module: "Store A" is offering a sale this weekend on a toolbox that runs 

7 for 29.99. The toolbox contains 

8 IVR module: Thank you for holding. We found the following movies for you. 

9 (Demographically Tailored Movie List Starts Here) 

10 IVR module: Darkness Falls (horror) 

11 IVR module: Red Dragon (Violence) 

12 IVR module: How to Lose a Guy in 10 Days (Romance) 

13 Since the contact 102 indicated a lack of interest in cosmetics but an interest in 

14 workshop tools, the probing dialog classifier 1 16 assigns the contact's 102 male 

15 gender confidence score to a value greater than that assigned to the contact's 102 

16 female gender confidence score. Since the contact's 102 male confidence score is 

17 higher, the IVR module 108 presents the contact 102 an advertisement and orders the 

18 list of movies in way likely to be preferred by a male (i.e. a tool advertisement and a 

19 list of movies where horror and violence titles come before romance). If however, 

20 the contact's 102 female confidence score was higher, the IVR module 108 would 

21 present the contact 102 an advertisement and order the list of movies in way likely to 

22 be preferred by a female (i.e. a cosmetic advertisement and a list of movies where 

23 romance titles come before horror and violence titles). 
24 
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1 Multiple Choice Classifier 

2 A multiple choice classifier 1 18 presents the contact 102 with one or more 

3 multiple choice questions retrieved from the dialog database 1 10. The multiple choice 

4 questions present a selection of choices related to one or more demographic 

5 characteristics that need to be resolved with respect to the contact 102. The multiple 

6 choice classifier 118 compares the contact's 102 responses to a predefined body of 

7 responses generated using neural net techniques and assigns a set of confidence scores 

8 to each of the demographic characteristics. The multiple choice classifier 118 stores 

9 the confidence scores in the demographic data 120 portion of the contact database 

10 112.^ 

11 The multiple choice classifier 118 is likely to be quicker than the probing 

12 dialog classifier 116, since a single multiple choice question can replace several yes- 

13 no probing dialog questions. In alternate embodiments of the present invention 

14 multiple choice questions can be hierarchically ordered, which is relevant to, for 

15 example, binary classification trees. 

16 An example of a multiple choice dialog is shown below. In this example like 

17 the one above, the IVR module 108 needs to know the contact's 102 gender before 

18 responding to the contact's movie theater inquiry. For the purposes of this example, 

19 the contact's 102 demographic characteristics are inferred from the confidence scores 

20 assigned by the multiple choice classifier 118 only. 

21 IVR module: Please hold while we retrieve information about movies in the 

22 theater you selected. 

23 (Advertising related "Multiple choice questions" Start Here) 

24 IVR module: While holding we will provide you with some existing 

25 promotions in your neighborhood. 
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1 IVR module: Please select which promotions will be of main interest to you. If 

2 you are interested in "Cosmetics" please say "Cosmetics". If you are interested 

3 in workshop tools, please say "tools". 

4 Contact: Cosmetics 

5 IVR module: Please choose from the following: a) lipsticks, b) eyeliner, c) 

6 foundation. Or say exit if you feel you have reached a wrong menu. 

7 Contact: eyeliner 

8 ("Multiple choice questions" End Here) 

9 (Demographically Tailored Advertisement Starts Here) 

10 IVR module: . . .promotions. . . 

11 (Demographically Tailored Movie List Starts Here) 

12 IVR module: Thank you for holding. We found the following movies for you. 

13 IVR module: How to Lose a Guy in 10 Days (Romance) 

14 IVR module: Darkness Falls (horror) 

15 IVR module: Red Dragon (Violence) 

16 Since the contact 102 indicated an interest in cosmetics and eyeliner, the 

17 multiple choice classifier 118 assigns the contact's 102 female gender confidence 

18 score a value greater than that assigned to the contact's 102 male gender confidence 

19 score. Also, since the contact's 102 female confidence score is higher, the IVR 

20 module 108 orders the list of movies in way likely to be preferred by a female (i.e. 

21 listing romance titles before horror and violence titles). If however, the contact 102 

22 had selected "Exit," the multiple choice classifier 118 will lower the contact's 102 

23 female gender confidence score. 

24 Note that experimental testing suggested that the gender questions presented in 

25 the example dialog above provide for strong gender prediction. 
26 
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1 While the acoustic classifier 1 14 preferably runs continuously in the 

2 background as the dialog between the contact 102 and the IVR module 108 is 

3 occurring, the probing dialog classifier 116 and the multiple choice classifier 118 are 

4 preferably not called unless the IVR module 108 needs the contact's demographic data 

5 and the acoustic classifier's 114 demographic characteristic confidence scores are less 

6 than definitive, such as if the acoustic classifier 1 14 sets the contact's 102 female 

7 gender confidence score to p=.45 and the contact's 102 male gender confidence score 

8 to p= 55. 

9 In an alternate embodiment, the probing dialog classifier 116 and the multiple 

10 choice classifier 118 operate in parallel with the acoustic classifier 1 14 and the dialog, 

11 so that when the IVR module 108 needs the contact's demographic data, the 

12 confidence scores of each of the classifiers 1 14, 1 16, and 118 can be combined to 

13 yield a more accurate demographic confidence score. 

14 Preferably, the probing dialogs and multiple choice questions are presented to 

15 the contact 102 during those periods when the contact 102 is not interacting heavily 

16 with the IVR module 108, such as when the contact 102 would otherwise be placed on 

17 "hold". 

18 Demographic characteristics stored by the various classifiers 1 14, 1 16, and 118 

19 in the demographic data 120 portion of the contact database 1 12 are preferably 

20 synchronized to ensure that the confidence scores from each of the different classifiers 

21 1 14, 1 16, and 118 refer to the same contact (or call instance). 
22 

23 Demographic Data Combiner 

24 A demographic data combiner 122 retrieves the confidence scores stored in the 

25 demographic data 120 for the various demographic characteristics. The combiner 122 

26 calculates a combined confidence score for each of the demographic characteristics. 
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1 The combined confidence score can be calculated in several different ways depending 

2 upon how statistically independent the different classifiers are 1 14, 116, and 118 and 

3 the demographic characteristic being identified. 

4 One calculation method uses an equal-weighted product combination. The 

5 equal weighted method equally weights and multiplies together the confidence scores 

6 from each classifier 1 14, 1 16, and 1 18 to yield the combined confidence score. 

7 Another calculation method is the weighted linear summation, according to the 

8 following formula: 

N 

9 S(C.) = ^ r .p .. (N is the number of classifiers, Ci is the i'th class, Pij is the 

1 j=\ J V 

10 probability of belonging to Ci as decided by Classifier j, and rj is trained 

11 weights) 
12 

13 A third calculation method is the weighted exponential, according to the 

14 following formula: 

15 w,-)=n^ 

16 A third calculation method uses a neural net, such as Multiple Layer 

17 Perception (MLP) network, with { p n , p i2 p iN } as the input. 
18 

19 The combiner 122 dynamically adjusts each classifier's 1 14, 1 16, and 118 

20 weight using ground truth data. The ground truth data may include product, service, 

21 or error-cost criteria, such as: a purchase value of a product or service being presented 

22 to the contact 102 by the call handling system 104; the degree to which various 

23 demographic audiences would understand the jargon used in presenting the product or 
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service; the costs of misidentifying the contact's 102 demographic information; and 
how much the contact 102 might spend on the product or service. 

The combiner 122 also dynamically adjusts a first classifier's 1 14, 1 16, or 1 18 
weight for a given demographic characteristic if the first classifier's confidence score 
differs from a second classifier's 1 14, 1 16, or 1 18 confidence score for that given 
demographic characteristic by a predetermined amount. For example, if the contact 
102 turns out to be female, and the acoustic classifier 114 identified the contact 102 as 
male, but the probing dialog classifier 116 and the multiple choice classifier 118 
identified the contact 102 as female, then with respect to the gender demographic, the 
combiner 122 would reduce the acoustic classifier's 114 weighting and increase the 
weightings for the probing dialog classifier 116 and the multiple choice classifier 118. 
Classifier weighting can be varied in a number of ways, including by 1/M % where M 
is a total number of contact dialogs to date. 

One exemplary embodiment of how the demographic data combiner 122 
combines the confidence scores generated by the classifiers 114, 116, and 118 using 
the equal-weighted product combination method is shown in the table below. Note in 
this example the classifiers 114, 116, and 118 are assumed to be independent since 
they use different techniques for identifying the gender demographic. 



Set of 
Demographic 
Characteristics 


Acoustic 
Classifier 
Score 


Probing Dialog 
Classifier 
Score 


Multiple Choice 
Classifier Score 


Demographic Data 

Combiner 
Classification Score 


Male 


0.4 


0.7 


0.5 


0.4x0.7x0.5=0.140 
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Female 


0.5 


0.2 


0.3 


0.4x0.2x0.4=0.030 


Don't Know 


0.1 


0.1 


0.2 


0.1x0.1x0.2=0.002 



1 

2 If the classifiers 14, 1 16, and 118 have different weightings, the combined 

3 confidence score in the last column of the table is altered according to the following 

4 formula: Hpi 1 ^, where pi is the p-value for each classifer i, and Ki is the corresponding 

5 power of the classifier, which is dynamically updated. Those skilled in the art 

6 however will recognize that alternative methods to the power-based one presented are 

7 also possible. 

8 From the table, the male demographic characteristic has a highest combined 

9 confidence score. Note that reliance on only the acoustic classifier 1 14 would have 

10 yielded an opposite gender determination. A "percent maleness" of the contact 102 is 

11 calculated according to the following equation: "percent maleness" = 

12 ((0.140)/(0.030+0.140))xl00% = 82.35%. 
13 

14 The IVR module 108 instructs one or more of the classifiers 1 14, 1 16, and 1 18 

15 to continue classification of the contact's 102 demographic characteristics until either 

16 a predetermined set of demographic characteristics have either individually or 

17 combined confidence scores that are above a predetermined threshold or until a 

18 predetermined time period has expired. For example, the IVR module 108 may 

19 instruct the multiple choice classifier 1 18 to continue asking the contact multiple 

20 choice questions until either the female or the male gender confidence score reaches at 

21 least p=.80 or until the contact 102 has been asked questions for 5 minutes. 
22 
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1 The combiner 122 stores the combined confidence score for each of the 

2 demographic characteristics in the demographic data 120, The IVR module 108 

3 retrieves the combined confidence score from the demographic data 120. 
4 

5 The IVR module 108 tailors information presented to the contact 102 using the 

6 combined confidence scores. More specifically, the IVR module 108 identifies a sub- 

7 set of the demographic characteristics having combined confidence scores exceeding a 

8 predetermined set of thresholds. Preferably, the set of thresholds corresponds to those 

9 demographic characteristics having a highest combined confidence score within each 

10 demographic characteristic category. For instance, if the demographic characteristics 

11 required for the contact 102 included gender (i.e. male or female) and a stress level 

12 (i.e. low stress or high stress) and if the combined confidence scores were 

13 respectively, p=J male, p=.2 female, p=.4 low stress, and p=.3 high stress, then the 

14 sub-set of demographic characteristics identified by the IVR module 108 for the 

15 contact 102 would be, male and low stress. The IVR module 108 then presents the 

16 contact 102 with information specifically directed to contacts having the sub-set of 

17 demographic characteristics (i.e. male and low stress). 

18 In another example, the IVR module 108 can use the combined confidence 

19 score to make a determination between presenting the contact 102 with a set of man- 

20 specific content, a set of women-specific content, or a set of gender-neutral content. 

21 Gender-neutral content could be played to a contact whose combined confidence score 

22 are still less than definitive, but gender specific information could be played to 

23 contacts whose combined confidence scores are above a predetermined threshold. 

24 Thus using the combined confidence scores, the IVR module presents to the 

25 contact 102 a dialog portion that is more demographically tailored to the contact 102. 

26 Such tailoring thereby enables the IVR module 108 to make a more appropriate set of 
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1 decisions during the dialog with the contact 102, including selecting and presenting: 

2 advertisements, music, jokes, and so on. 
3 

4 Figure 2 is a flowchart of one embodiment of a root method 200 for extracting 

5 demographic information. In step 202, initiate a dialog between a contact and a call 

6 handling system. In step 204, select a set of demographic characteristics. In step 206, 

7 assign a set of acoustic confidence scores to the demographic characteristics. In step 

8 208, assign a set of substantive confidence scores to the demographic characteristics. 

9 In step 210, combine the acoustic and substantive confidence scores for each of the 

10 demographic characteristics. Then in step 212, tailor information presented to the 

11 contact using the set of combined confidence scores. Details of the root embodiment 

12 just presented are now discussed with respect to Figures 3 A and 3B. 
13 

14 Figures 3A and 3B are a flowchart of one expanded embodiment 300 of the 

15 root method for extracting demographic information. To begin, in step 302, a dialog 

16 between the contact 102 and the call handling system 104 is initiated. In step 304, the 

17 IVR module 108 retrieves and presents predetermined dialog segments from a dialog 

18 database 1 10 to the contact 102. In step 306, the IVR module 108 receives responses 

19 and information requests back from the contact 102 which are then stored in a contact 

20 database 112. In step 308, the IVR module 108 activates a set of demographic 

21 classifiers, including an acoustic classifier 114 and a set of substantive classifiers 115. 
22 

23 Acoustic Classifier 

24 In step 310, the call manager 106 diverts a copy of the contact's 102 speech 

25 signal utterances to the acoustic classifier 1 14 without interrupting the dialog between 

26 the contact 102 and the IVR module 108. In step 312, the acoustic classifier 114 
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1 extracts from the contact's 102 speech signal features relating to one or more 

2 demographic characteristics that need to be resolved with respect to the contact 102. 

3 In step 314, the acoustic classifier 1 14 compares the contact's 102 speech 

4 signal features to a predefined body of speech signal features generated using neural 

5 net techniques and assigns a set of confidence scores to each of the demographic 

6 characteristics. In step 316, the acoustic classifier 1 14 stores the confidence scores in 

7 a demographic data 120 portion of the contact database 1 12. 
8 

9 Probing Dialog Classifier 

10 In step 318, a probing dialog classifier 1 16 presents the contact 102 with one 

11 or more probing dialogs retrieved from the dialog database 1 10. In step 320, the 

12 probing dialog classifier 116 compares the contact's 102 responses to a predefined 

13 body of responses generated using neural net techniques and assigns a set of 

14 confidence scores to each of the demographic characteristics. In step 322, the probing 

15 dialog classifier 116 stores the confidence scores in the demographic data 120 portion 

16 of the contact database 112. 
17 

18 Multiple Choice Classifier 

19 In step 324, a multiple choice classifier 118 presents the contact 102 with one 

20 or more multiple choice questions retrieved from the dialog database 1 10. In step 

21 326, the multiple choice classifier 118 compares the contact's 102 responses to a 

22 predefined body of responses generated using neural net techniques and assigns a set 

23 of confidence scores to each of the demographic characteristics. In step 328, the 

24 multiple choice classifier 118 stores the confidence scores in the demographic data 

25 120 portion of the contact database 1 12. 
26 
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1 Demographic Data Combiner 

2 In step 330, a demographic data combiner 122 retrieves the confidence scores 

3 stored in the demographic data 120 for the various demographic characteristics. In 

4 step 332, the combiner 122 calculates a combined confidence score for each of the 

5 demographic characteristics. 

6 In step 334, the combiner 122 dynamically adjusts each classifier's (114, 116, 

7 and 118) weight using ground truth data. In step 336, the combiner 122 also 

8 dynamically adjusts a first classifier's (1 14, 1 16, or 1 18) weight for a given 

9 demographic characteristic if the first classifier's confidence score differs from a 

10 second classifier's (1 14, 1 16, or 1 18) confidence score for that given demographic 

11 characteristic by a predetermined amount. 

12 In step 338, the IVR module 108 instructs one or more of the classifiers 1 14, 

13 1 16, and 1 18 to continue classification of the contact's 102 demographic 

14 characteristics until either a predetermined set of demographic characteristics have 

15 either individually or combined confidence scores that are above a predetermined 

16 threshold or until a predetermined time period has expired. In step 340, the combiner 

17 122 stores the combined confidence score for each of the demographic characteristics 

18 in the demographic data 120. 
19 

20 In step 342, the IVR module 108 retrieves the combined confidence score from 

21 the demographic data 120. In step 344, the IVR module 108 tailors information 

22 presented to the contact 102 using the combined confidence scores. 



23 

24 While one or more embodiments of the present invention have been described, 

25 those skilled in the art will recognize that various modifications may be made. 
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Variations upon and modifications to these embodiments are provided by the present 
invention, which is limited only by the following claims. 
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